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This study assumes that multiple choice test items 
generally proidde the testee with several solutions, one of which is 
correct and the others of which are wrong. If pupils are unable to 
answer a question, one would expect that the wrong choices have equal 
chances of being selected. In many multiple choice items on 
stoichiometric calculation which have been studied over recent years, 
pupils were attracted to one or two wrong answers more than to the 
others. It is apparent that they have reached a wrong result by using 
a wrong answering strategy. The aim of this study was to develop new 
multiple choice test items involving stoichiometry in such a way that 
the correct and the two false answering strategies would lead to 
different results, to create items that include only number ratios 
which allow quick mental calculations and to determine whether on the 
new items pupils actually adopt false answering strategies. The 
strategy for developing questions and answer choices is discussed in 
detail. Instriunents were administered to pupils in grades 11 and 12. 
This study concludes that when developing a test for classroom or 
research, items should include false and correct strategy application 
in the same proportion. (CW) 
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Multiple choice items provide the testee with several solutions among 
which he has to decide. As a rule there is only one correct answer and 
all the other answer options - the distractors - are wrong. If testees who 
are not able to answer the question were to decide on one of the answer 
options by chance, all of the distractors should be selected with the same 
frequency. In many of the multiple choice items on stoichiometric calculation 
which we have studied over recent years, however, testees are attracted 
to one or even to two distractors. From the explanations given by testees 
on multiple choice items for their answers in paper-and-pencil tests or in 
interviews, it is apparent that there is a system in the way they have reached 
the wrong result: they use their own answering strategies, which may well 
be quite cleverly thought out, but which do not quite correspond with the 
thinking of chemists, '^his matter can be clarified using an example. 



Test item 16,341 

'^he chemical formula of sulphur dioxide is SO2. How many g of sulphur 
would be found in 6 g of sulphur dioxide? 

4 g (A) / 3 g (B) / 2.5 g (C) / 2 g (D) 



This item was given in the Federal Republic of Germany to a large number 
of grammar school pupils in the 10th to 13th grades. The correct answer 
is option B, many pupils however opted for distractors A and D (Fig. 1). 
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From the pupils* comments it is possible to reconstruct the considera- 
tions which led them to these distractors. Many pupils who arrived at distrac- 
tor D argue als follows: 

0 occu/i^ La doudCa the amourti ojL S fdacau^ie the ^onjnala 
SO2). If. 2 g 0^ S OAC pn^d^aai, iioica much 0 ^6 p/ia^c/ii, a. ^ g." 

These pupils precede from the the false asssumption that the amount ratio 
n(S) : n(0) can be equated with the mass ratio m(S) : m(0) 

(1) n(S) : n(0) = m(S) : m(0) 

Pupils arriving at distractor A argue along different lines: 

"7ha atomic ma':>^ of. OKygcn 16 a and of mlphiui 32 a, thai -l^ 
twice a 3 high a^^ OKygaa. /hay aic thazefo^a in a aatio of 2 : 7. Con- 
6aquantCy 6 g of ^ulphu/i dioxide contain 4 q of ^iLiphiui." 

These pupils equate the molar mass ratio iVl(S) : M(0) with the mass ratio 
m(S) : m(0): 

(2) M(S) : iVl(O) = m(S) : m(0) 

Similar pupil comments were obtained when pupils were handed only the 
item stem as an open-answer question. 

It is interesting and important to note that the answer patterns for 
many items which have been used abroad can also be explained using the 
hypotheses expressed in equations (1) and (2). This justifies the assumption 
that the false strategies which have been outlined are also widely used 
abroad. 



Problem 

An obvious step is to take the false strategies used by pupils in dealing 
with items on stoichiometry as a basis for constructing new multiple choice 
items. The aim of this study was 

/!/ to develop new items in such a way that the correct and the two false 
strategies would lead to different results {i. e. to different answer 
options), 
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HI that items include only numbers or number ratios which allow a quick 
mental calculation of the answer, 

/3/ to determine whether on the new items pupils actually adopt the false 
strategies on which they are based* 



Procedure 



Instruments 

If in solving multiple choice questions on stoichiometric calculation 
the corrv^ct and any false strategy are to lead to different answer options, 
it is necessary that the figures included in the item should be such that 
both elements of the binary compound differ from each other with regard 
to the amount ratio (nj : n2), the molar mass ratio (Mj : and the mass 
ratio (mj : m2). The relationship existing between the amount, the molar 
mass and the mass is expressed in 

(3) "1 • "2 "ij/Mj : "^2^'^2 

One of the false assumptions described above involves pupils* failing 
to distinguish between the ratio of molar masses and of amount, or of molar 
masses and masses 

(4) Mj : M2 = rij : n2 or Mj : M2 = mj : m2 

In the other misconception which has been described they equate the 
mass ratio with that of amount of substance: 

(5) mj : m2 = rij : n2 

Let us now attempt to discover "by hand", so to speak, what compound 
and what further information are appropriate for a test on stoichiometry. 
For this purpose we will try out in turn various conditions* 

For a compound of the type AB nj = n2^ Should the elements A and 
B of this compound in additon share the same molar mass, then also 



ERLC 



- 5 - 



Mj = ^^^^^ these boundary conditions the amount ratio, the molar mass 
ratio and the mass ratio are the same, i. e. although one or the other of 
our false strategies is used, the correct answer is still obtained. 

It is not much better when an AB-type compound is selected where, 
however, the molar masses of elements A and B do differ. In this case the 
ratio of molar masses and the mass ratio coincide. Here too using a false 
strategy may still produce the correct result. This applies also in the situ- 
ation where the molar masses of elements A and B are the same, but the 
formula of the compound does not reveal the simplest form AB. Under these 
boundary conditions the ratios of amount and mass are identical. 

Let us now test whether a formula of the type AB2 produces a better 
result. In this case nj : n2 = 1 : 2. If in additon we make Mj : M2 = 2 ; 1, 
it also follows that mj ; m2 = 1 : 1. We now look for concrete chemical 
elements for A and B whose molar masses are in a ratio of 2 : 1 and which 
combine as an AB2-type compound. These should be compounds which do 
actually occur. We can consider here MgC2, SO2 and CUS2. These formulae 
can be inserted into any number of item texts. Following the pattern of 
item 16,341, what is still missing is the value in the item stem with which 
the mass is to be calculated. This value can be set according to various 
principles. If one assumes that subjects calculate amounts by way of the 
quotient of mass and molar mass, it makes good sense to ensure that numeri- 
cal values are used for this quotient which result in a whole number of 
a fraction. It is also conceivable that a correct answer could also be found 
by way of a ratio as is shown under (3). In this case an easily calculable 
figure can be obtained by taking as mass the "least common multiple" of 
(nj + n2) in equation (3), of (Mj + M2) in equation (4) and of (mj + m2) 
in equation (5). In this case v/e obtain (1 + 2) x (I + 1) = 6. With that we 
have now rediscovered item 16,341 with option B as the key and A and 
D the two distractors. Distractor C is not associated with any misconception 
and is included simply as an intermediate value. 

It is however also possible to incorporate one of the formulae deve- 
loped and the mass value of 6 g into the text of an item in such a way 
that the masses of elements A and B are given and the required formula 
is asked for. The item would then have the form: 
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Test item 00.001 




6 g of an oxide of sulphur contain 3 g of sulphur, 


the remainder being 


oxygen. Which formula fits this statement? 




SO (mj : m2 = nj : n2) 


(A) 


(Mj : M2 = nj : n2) 


(B) 


SO2 (mj/Mj : m2/M2 = nj : n2) 


(C) 



The brackets contain the steps of the calculation. They do not of course 
appear in the text of the item. 

It is very unlikely that pupils will accept such improbable formulae 
as SO and S2O as genuine alternatives to SO2. If this type of item is to 
be retained, the element symbol S should be replaced by the symbol for 
the unknown element X with the molar mass of 32 g/mol. 

The formula CuS2, on the other hand, can be included in both question 
types: 



Test item 00.002 








The chemical formula 


of a 


copper sulphide is CuS2. 


How many g of 


copper would be found 


in 6 g of copper sulphide? 




2 g (nj 


: "2 = 


= mj : m2) 


(A) 


3 g (nj 


X Mj : 


"2 ^ ^2 "^1 • "^2^ 


(C) 


4 g (Mj 


: M2 


= mj : m2) 


(D) 



In this example a fourth answer option needs to be added, e. g. the inter- 
mediate value 2.5 g as distractor B. In the following item the formula is 
not given but asked for: 
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Test item 00,003 




6 g of a sulphide of copper contain 3 g of copper, 
.'julpiiur. Which, formula fits this statement? 


the remainder being 


CuS (mj : m2 = nj : n2) 


(A) 


CuaS (Mj : M2 = nj : n2) 


(B) 


CuSa (m,/Mj : m2/M2 = nj : n2) 


(D) 



Here also a fourth answer option, e. g. the formula CU2S3 as distractor 
C, should be added. A very similar '•:em has been received from an Exami- 
nation Board in Great Britain and has been discussed in some of our publi- 
cations. Consequently it has not been reused in the present study. 

The previous discussion has been intended to clarify the principle on 
which we have constructed new multiple choice items on the basis of fami- 
liar answering strategies. It is obvious that a whole range of items can be 
obtained if for (5), (6) and (7) we insert simple number ratios like 1 : 2, 
1 : 3, 2 : 3 etc. in various combinations. In order to "play through" systema- 
tically the different possibilities, a colleague of mine has designed a computer 
program. As I myself am not a mathematician I will not go any further 
into the computer calculations. 

It is a matter of course that our insights into pupils* false answering 
strategies enable us as well to construct items in such a way that the pupils 
are meant not to adopt the above described wrong strategies. We have to 
decide between two possibilities: 

/I/ If a chemical formula is given, it cannot be avoided that the calcu- 
lation is made under the false assumption that the amount ratio n^ : 
and the mass ratio m^ : m2 are identical. The only thing you can do is 
to find a particular compound with chemical elements of extremely different 
molar masses so that the molar mass quotient Mj : M2 results in a very 
high or low value and, moreover, appears as a so-called "dirty figure''. The 
question is if this procedure actually keeps pupils from equating Mj : M2 
with m, : m^. 



ERLC 



8 



- 8 - 



12/ Ii is very much easier lo construct test items with little possibilities 
for testees to use wrong strategies if a chemical formula is to be calculated. 
In this case the mass ratio m^: and the molar mass ratio Mj : M2 
of the elements involved should also produce extreme values and dirty figures 
in order to confront the very pupil who calculates the amount ratio as a 
quotient of mj : m2 or Mj : M2 with a completely unusual formula, A good 
example for such a formula is CI3r^ with Mj : M2 = 12 : 80 = 3 : 20 and 
mj : m2 = 12 : 320 = 3 : 80, Hardly any pupil will accept a formula like 
^12^^80 ^3^^80 rounded off, CBr^ and CBr^Q. 

From the many items which we have developed and tested 1 would 
like to present to you just two as examples. This is the first one: 



Test item 50,638 








The chemical formula of a 


compound is XS2. 


What mass 


of X combines 


with 16 grams of sulphur 


in this compound? 


The molar 


mass of X is 


48 g/moK 








8 g (nj : n2 


= mj : m2) 




(A) 


12 g (n, X M, 


; n2 X M2 = mj 




(B) 


24 g (M, : M2 


= mj : m2) 




(C) 


32 g 






(D) 



As we know only two false strategies, so only two distractors can 
be calculated. Di',tractor D was added as an appropriate-looking complement. 
This is the second test item: 
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Test item 50,644 

40 g of a compound contain 16 g of sulphur, the remainder being molyb- 
denum* Which formula fits this statement? 

M0S2 (mj/Mj : m^/M^ = : n^) (A) 

M03S2 (mj : m2 = nj : n2) (B) 

M02S (C) 

.VI03S (M, : M2 = n, ^ n^) (D) 



Distractor C was added as an appropriate-looking complement. 



Sample 



At the beginning of the 1987/88 school year sets of test packages 
each for 30 pupils were sent to 21 grammar and vocational school teachers 
who had agreed to conduct the test. 17 teachers actually distributed the 
tests in classes. The returns from 353 pupils were useable for our evaluation. 
This figure is made up to 88 % of pupils in the senior level of grammar 
school, most of whom were in grade 11 or were taking part in Basic Courses 
in chemistry in grade 12. The remainder were attending vocational schools. 
51 % of the subjects were male, 48 % were female. Neither the teachers 
nor the pupils were then chosen at random so that the sample cannot be 
regarded as representative in a statistical sense. 



Data Processing 

All the relevant information given by pupils on the personal question- 
naires (age, school class, course etc.) and all the data which could be taken 
from the completed answer sheets (selected answer option, etc.) were coded 
and fed into a computer. Subsequently a frequency calculation was performed: 
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for each item the percentage of pupils selecting each answer option, or 
not answering, was calculated. From the explanations which accompanied 
the pupils' calculations it was attempted to reconstruct their answering stra- 
tegies. Relevant sections of these explanations were put on file. 



Results 



Fig. 2 shows pupils' response rates for each of the answer options 
on item 50,638. This item was answered by H pupils. Distractor C is clearly 
the most attractive. 

Fig. 2 

Answer profile for question 50,638 
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Pupils' comments provide confirmation of the two false strategies on 
which the construction of the item is based actually being used. Here are 
two sample comments: 
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"rJi ihv cc m/Kuml has ihc /^itmiUu XS, and cvn*,cqucniOj iJxvn i/uic 
aw iuHcc «/> /;/<7m/ pa'ds S a^ patis V U> U /fund, an amiuni (./ <S r/ c/ 
A' musi Lc p7C)cni mcc S v J = 76 

This pupil draws on ihe siraiegy ouilined under (5) and is led lo di.sirac- 
lor A, The next pupil uses ihe sirategy described in equation (4), He writes: 

tS a mt C V I () if ^, „ 
V = — ; — -i r q 

Following this path iie arrives at disiractor C. 

In item 50,644 13 is the most attractive distracior (fig. 3), 

Fig. 3 

Answer profile for question 50,644 
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This item was answered by 91 pupils. Here too the comments provide 
evidence that the main error was a matter of using two false strategies. 
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Here are two sample comments: 
^tO g 0^ ihd compound: 

moiyidcntMU '>u Iphan, 

24 cj U cj / : U 

l.'^ g 1 g 

7ha riaVw o/ buiphuA to molyidanwn i^b 2 : 5," 

This pupil has drawn on equation (5) and selected distractor B. From 
ihe next comment it can be seen that the strategy expressed in equation 
(4) has been used. This has led to distractor D. 

"Rdiativa atomic ma^^ o/ molyddanwn: 96 g/mol 
Ralativa atomic ma-^^ o/ 6u£phiui: 32 g/mo£ 



Ratio 



6alphiui )2 g/mot 



Trial has therefore led to the expected result. It has however also 
produced one surprise. Let us look again at the answer profile for item 
16,341 (fig. 1). For all the items we have ourselves tested, the distractor 
which has been most attractive has previously always been that based on 
strategy (5), the failure to distinguish between the ratios of mass and amount. 
This applies, for example, also to item 50,644 (fig. 3). With item 50,638 
an item has however now been added for which the most attractive distractor 
is clearly the one based on strategy (4), the failure to distinguish between 
the ratios of molar mass and mass (fig. 2). Similarly in the items from Great 
Britain, for which the answer profile is known to us, it was always the dis- 
tractor associated with strategy (4) which proved most attractive. Whenever 
we have distributed tests on stoichiometry, the set of items has always 
had appended to it as the final sheet a list of the molar masses of familiar 
elements. In the tests from Great Britain the molar masses were included 
in the very text of the item. Our hypothesis: the majority of pupils try 
to make do with those numbers which appear in the stem of the item. How- 
ever, they also try to make do with as few numbers as possible. 

Perhaps what happens when pupils attempt an item on stoichiometry 
is that they allow themselves to be led by the number ratios and only later 
attribute to them any chemical significance. If this is the case, pupils would 
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be acting like good scientists. When Avogadro developed the law which bears 
his name he was at first impressed by the simple proportions which had 
been found in gas reactions. As he attempted to bestow an interpretation 
on these simple proportions his law was formed. The periodic system of 
elements was developed in a similar way. 

The explanation which has been given for the different answer pro- 
files strikes us as extremely plausible, but it has at present only the status 
of a provisional hypothesis as the empirical basis for its support is still 
too small. 



Conclusion 

Items on stoichiometric calculations can be developed in different ways: 

/I/ If a chemical formula is to be developed, the test can be construc- 
ted in such a way that pupils can actually make the errors they are keen 
on making. But it is also possible to keep them from error-making. 

IV If, however, a chemical formula is given and masses are to be 
calculated, pupils can always draw on the false strategy nj : n2 = mj : m2- 
Perhaps the answering strategy Mj : M2 - mj : m2 can be made less attrac- 
tive. 

In chemistry lessons the different tests described above may be used 
for individual purposes. It is very important that in chemistry lessons pupils 
think about the errors they are so keen on ni^King. In order to maintain 
awareness of the cause of the error, problem areas must be reviewed during 
lessons on a regular basis and in various guises. 

When adopting norm-referenced tests for assessment or analyzing treat- 
ment effects in empirical research, the individual items should allow fair 
comparison on equal level. In classroom tests for example pupils should be 
given the two different types of items - false strategy applicable/not appli- 
cable - in the same proportion. 

Consequently, a great variety of tests is required. In this study it has 
been revealed that it is very easy to develop items on stoichiometric calcu- 
lation. 




